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Introduction 



This research area is intended to contribute towards the development of the 
next generation of remotely operated vehicles (ROV’s) currently under development 
by the US Navy. A concept for one such design is shown in figure 1. In these 
vehicles, the operator does not operate traditional controls (switches, joysticks, 
levers) in order to operate the vehicle, but experiences complete visual, auditory, 
force, and touch feedback directly from the remote environment in which the vehicle 
is operating. If the fidelity of the sensory feedback is high enough, the operator 
experiences ‘telepresence’ or ‘virtual reality’, a state in which the operation of the 
vehicle become more natural to the operator, thereby reducing operation time and 
increasing task effectiveness. 

It is known that many, if not most, diving operations axe conducted in conditions 
of little or no visibility, and human divers accomplish much of their work based on 
what they can feel rather than what they can see. It follows that if the ROV has to 
operate in similar conditions, we need to know how this blind touch probing would 
be perceived by the operator on the surface, and what strategies would be employed 
in order to accomplish certain tasks. It is in this area of probing by means of touch, 
commonly called haptic probing, that the PI is involved. 

This final report covers research into several aspects of haptic sensing, and in 
particular, investigations of the way the haptic system may be used in conjunction 
with a teleoperator system to identify remote objects. The work proceeded in 
two main directions. Firstly, experimental data obtained by the PI while working 
as an ONR Fellow at NOSC was analysed, and several hypotheses regarding the 
mechanics of human haptic probing were evaluated. In the second area, the design 
of a multi- degree of freedom mechanical hand is undertaken. Such a device will be 
required to achieve the necessary dexterity expected of a system capable of achieving 
telepresence on the part of the operator. 

Haptic Probing 

In the summer of 1989, the PI had the oportunity to work with the Adaptive 
Systems Branch, NOSC in order to begin preliminary work in this area. The branch 
had a CRL force reflecting telemanipulator (see fig 2) on which experiments were 
performed. In the ten weeks allocated to the fellowship, only preliminary exper- 
imental procedures could be designed and executed. In the fall of 1989 the PI 
researched some of the physiological background to this subject and proposed sim- 
ple models for the operation of the human haptic system when used in conjunction 
with a bi-lateral force reflecting telemanipulator. These findings have been writ- 
ten up in one conference paper (appendix A) and one journal paper (appendix B), 
currently accepted for publication, though not yet in print. 
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In order to continue this research, a CRL manipulator has been purchased and 
will be delivered in December 1990 to NPS. This will allow further extensive testing 
of hypotheses on haptic probing in conjunction with telemanipulator devices. It 
is expected that this work will lead to a basic understanding of the mechanics of 
haptic probing and object recognition, and the development of computer assisted 
aids to enhance operator recognition. 

End-Effector Design 

Figure 1 shows that the remote robot arm is fitted with a human-like (anthro- 
pomorphic) end effector capable of manipulating hand and power tools. Although 
such devices exist in the laboratory, their mechanical design is somewhat complex 
due in part to the remote actuation of the finger joints by means of tendons. This 
leads to frequent failures in heavy use of the hand. The PI has developed a novel 
approach to the design of such hands in which the actuator, consisting of a minia- 
ture DC motor and gearbox, is built directly into the finger unit itself. A prototype 
finger based on this design is shown in figure 3. Part of the project was to further 
develop the design of this end effector to decrease the size shown in figure 3 to one 
which approaches that of the human hand. 

Commercial gearmotors were identified for this task and a prototype finger joint 
was constructed. This reduced the size from about 3 x human size to approximatly 
1.5 x human size. Appendix 3 includes some of the drawings on which the new 
design is based. 

Unfort unatly, work in this area had to be suspended until FY1990 due to the 
redirection of funds towards the purchase of the CRL telemanipulator mentioned 
previously. Work has re-started on this topic with a graduate student investigating 
the control system for the finger unit. 

Conclusions 

Good progress has been made in both the areas described, although it has taken 
a year to establish a laboratory facility in which the work can be performed. With 
the delivery of the CRL arm imminent, and several graduate students interested in 
many aspects of the research, progress is expected to continue in 1991. 
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FIGURE 2 FORCE REFLECTING TELEMANIPULATOR 
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Abstract 

In this work, we are concerned with the role that the haptic system plays in teleoperation 
i.e. the exploration and manipulation of objects by a human operator using a remote robot 
arm. The approach is to incrementaly enhance the remote touch-sensing capability beyond 
kinesthetic force feedback to include contact data and local re-perception and compare the 
time to identify quasi-two dimensional objects with that of a directly held probe. Results 
obtained indicated that friction between the remote probe and the environment made the 
feedback signal “noisy” leading to conflicting and inaccurate hypotheses by the operators. 
Sensory feedback improved the signal to noise ratio giving performance levels approaching 
those of direct, as opposed to remote, probing. 

Introduction 

In this paper we axe interested not so much in direct manipulation of objects, but in 
remote manipulation using a mechanical teleoperation device. The placement of a telema- 
nipulator between the operator and the task to be performed acts a s a kind of filter to 
many kinds of sensory feedback, including touch, and in most cases degrades the operator’s 
ability to accomplish tasks that would be quite simple if performed directly. Nevertheless, 
the study of telemanipulative tasks is of considerable interest since this is a common way of 
separating humans from hazardous environments. 

Contemporary telemanipulation facilities usually contain many sensory feedback paths 
to assist the operator in the accomplishment of some task. The manipulator itself may 
have force-feedback capability allowing the operator to sense the presence of an obstacle 
or constraint. Force feedback is usually of the terminus type, as shown in Figure 1, where 
only end-point forces at the control handle are felt, although some manipulators implement 
anthropomorphic feedback in which shoulder, elbow, wrist, hand and Anger forces are sensed 
through an exo-skeletal structure worn by the operator. Visual feedback is obviously im- 
portant, and this may be accomplished either directly through a viewing window, or by 
using a camera and video display for more remote applications. Audio and touch feedback 
arc also significant contributors to performance and we complimentary in many ways, since 

sound produced by tapping an object provides confirmation of contact in the absence of 
direct touch sensing by the remote probe. For the work reported here, particular attention 
is focussed on the sense of touch. The reason for this is, firstly, that many fine motion 
manipulations, such as assembling an electrical connector, utilize touch feedback more than 
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Figure 1: Operator Handle on Terminus Tele- Manipulator 

used to imply greater meaning than Gibson’s [2] “purposive touch” contained in his original 
definition. We define the haptic system to comprise many parts including: 

1. Tactile (localised) sensing of fine features 

2. Proprioceptive (kinesthetic) sensing of coarse position 

3. Other sensing systems such as temperature and pain 

4. A two-way communication channel between the central nervous system and the brain 

5. Perception processes to formulate hypotheses about the environment 

6. Motor control mechanisms to re-distribute the primary sensor systems 

Previous work in this area has concluded that although the haptic system may be used 
to recognize three dimensional objects both accurately and quickly, two dimensional object 
recognition is accomplished less successfully. In the tests reported by I/cdcrman, Klatzky 
and Barber |4j, raised two dimensional profiles were traced and recognition was attempted, 
resulting in poor performance. In other work on three dimensional object recognition re- 
ported by Klatzky, Lederman and Metzger [3] blindfolded subjects were allowed to pick up 
one of a set of 100 common objects and attempt recognition. In these tests, good results 
were obtained with only 4% of the tests resulting in mis-classification. Lederman, Klatzky 
and Bajcsy (4] report other published work which supports their conclusions that the haptic 
system is poorer at recognizing two dimensional shapes compared to its abilities regarding 
three dimensional objects. 
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Figure 2: Independent Variables of Haptic Perception 

In the work reported here, we have investigated further the abilities of the haptic system 
to explore and recognize essentialy two dimensional shapes using a telemanipulator. In ad- 
dressing the problem of emulating the human haptic system in machines, much attention has 
been focussed on the area of tactile sensing. Machine haptics obviously has contributions 
from a much larger collection of components than just sensing. By considering the current 
status of teleoperation, and the efforts to improve this status so that the operator actually 
feels as though he or she is in the remote workplace (telepresence), some of the other mecha- 
nisms which impact teleoperator performance can be identified. Our research has identified 
four such mechanisms, which we propose to define as haptic variables. They directly in- 
fluence the ability of teleoperators to perform complex tasks. These haptic variables are 
(1) tactile sensing (2) tactile display (3) force reflectance and (4) end effector dexterity. Al- 
though these mechanisms are also present in the human haptic system, they are less variable 
for the purposes of experimentation. These independent variables are represented in Figure 
2, which also indicates examples of discrete componenents of each technology beginning with 
the mofit simple in the periphery of the diagram and increasing in complexity towards the 
center, which is the goal of full telepresence. A system comprising components located near 
the center of the diagram would be expected to have capabilities approaching those of the 
human hand. Note that for mechanical teleoperator systems with man in the loop control, 
the definition of haptic variables is consistent with the definition of the components of the 
haptic system given earlier, since the human operator provides the information transmission, 
cognitive and motor control functions directly through the manipulator. 

Method 

The objective behind the experimental work reported in this section Is to investigate 
object recognition through remote haptic probing alone, and to determine which haptic 
variables will produce the rr¥>*t significant improvement in performance. In selecting the 
object set that operators would be asked to identify, it was decided to use wooden letters of 
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Figure 3: Remote Tele- Manipulator Probe 



the alphabet about six inches tall and one inch thick. Fbr each trial, a letter was selected at 
random from the set of 26 and attached in a random orientation to a metal taskboard. The 
manipulator used was a CRL force reflecting unit of the terminus type, which means that 
only forces at the controller handle are sensed. Force reflection is arlnevrd rncrlinnicnlly 
through antagonistic cables operating each degree of freedom of the manipulator. The 
remote probe consisted of a 1/4 inch diameter steel tube about 6 inches long. Figure 3 
shows the manipulator probing a test character. The operator handle is constructed in the 
form of a pistol-type grip (see figure 1). For all tests operators wore earplugs and headphones 
connected o a pink noise source so that all audible cues were masked. The operator was 
prevented from directly viewing the taskboard and remote probe by a curtain. 

A total of three operators were used in each of four experiments. Fbr each experiment, 
some twenty trials were performed . Although each operator had prior experience using 
the teleoperator system, the trials for each of the experiments were performed in a random 
sequence to equalize the effects of increasing familiarity with the equipment over the course 
of the experiment. The four experiments all involved the recognition of a randomly selected, 
randomly oriented letter of the alphabet, but differed in the sensory feedback provided to 
the operator during his probing. The specific conditions relating to each experiment were 
as follows. 



Experiment 1: 
Experiment 2: 
Experiment 3: 

Experiment 4: 



Force feedback only, audio and visual masking. 

As above but with edge contact feedback, 

Same as Experiment 1, but with observation of hand 
controller movement permitted. 

No teleinanipulator, tracing using hand held probe 



Experiment 2 allowed the operator to observe when the remote probe was in contact with 
the edge of the letter. To achieve this, ea,ch wooden letter had aluminum foil tape wrapped 
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PERCENTAGE OF CORRECT IDENTIFICATIONS 



Figure 4: Experimental Hesnlts 

around the edge, and a battery connected to it. The other battery terminal was connected 
to the remote probe so that when the probe wa s in contact with the edge of the letter a 
circuit was completed and a small light emitting diode (LED) was illuminated. The LED 
was placed into a small hole drilled into the mask worn by the operator so that edge contact 
confirmation was made available to him directly in front of his right eye. Prior to each 
test session, the operator was allowed to view a copy of the font used for the characters so 
that subtle differences could be observed, such as the difference between an M and inverted 
W, or an N and a Z. Operators were asked to provide a running commentary of what they 
thought was happening during the test. Operator comments and remote probe activity were 
recorded onto the name video tape. The operator was instructed to emphasize accuracy of 
identification rather than speed, and to make a clear identification at the conclusion of the 
probing. Operators were told if they made an incorrect identification, and the test continued. 
A time limit of ten minutes for recognition of a single letter was enforced, and the number 
of trials resulting in non-identification after this period was recorded. A second performance 
measure was the elapsed time required for correct identification, which was then averaged 
over all trials for the experiment. 

Results 

The results are shown in Figure 4 which indicates both the average time for character 
identification and the number of times identification was not possible within the permitted 
time period. 

Discussion 

•The results of Experiment 1, together with the commentary provided by operators for 
the cases where no successful identification was achieved, provide insight into the process 
of teleoperation, and how performance may be enhanced. The main ambiguity observed by 
operators was the inability to distinguish contact forces between the probe and the letter 
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from frictional forces between the probe and the taskboard. This was observed during the 
tests when the probe appeared to be following an edge quite accurately, but when a corner 
occurred, the probe continued to move in a straight line along the taskboard, constrained 
only by the end-point frictional forces. The operator wan unaware of the situation until the 
length of the perceived edge lierame largrr than his a priori expectation* about the longrnt 
real edge, and a re exploration of the taskboard occurred. This lends In (he miuepl ,,f n 

signal to noise ratio for Imptb data where the signal In this case in the pi obe Icltri rohtmt 
force ami the noise is the probe taskboard frictional force. In situations mu h an Kxpeiioienl 
1 where the signal is relatively small compared to the noise, many ambiguities wise making 
the identification of simple primitives such as edges or corners riiflu nil. I'Wqnrntly, even 
contact itself was incorrectly assumed to have occurred and haptic probing took place totally 
in the presence of noise only. 

The generation of end-point frictional forces at the probe is closely linked to the levels 
of Internal frictional forces within the telemanipulator Itself, as Indicated by the following 
example. Suppose the control handle can be moved in any direction in space, but to cause 
such movement a static frictional force of say 5N has to be overcome. No obstacle in the path 
of the remote probe can be detected until a force greater than 5N is exerted. If one considers 
the general strategy of probing a letter, the operator attempts to conceptualize the location 
and orientation of the taskboard and then tries to move the probe lightly over the board until 
contact with the letter'is made. Then, still minimizing the contact force between the probe 
and the taskboard, the probe is traced around the letter building up individual primitives 
into features and spatially mapping them into a recognizable object. Light contact with 
the taskboard is required to minimize the generation of end point frictional forces thereby 
increasing the signal to noise ratio during the probing phase. The internal friction of the 
manipulator however determines the magnitude of the friction force at the probe tip through 
the equation of static friction*. 



F = iiR 

where F is the frictional force along the plane of the taskboard, fi is the coefficient of static 
friction for the pair of materials comprising the task board and probe, and R is the force 
applied normal to the task board by the operator. If P is the internal static frictional force 
of the manipulator in a direction normal to the task board, it can be seen that 

F > pP 

Higher internal frictional forces therefore require larger contact forces in order to dis- 
tinguish eml point frirtioii from object contact, and the search is cliaractc rised by clumsy, 
sometimes sudden movements when the probe leaves the object. Large excursions of the 
probe inhibit accurate relocation on the object and generally degrade the spatial mapping 
of primitives already identified. r 

Lower frictional forces allow delicate probing to take place, small, subtle features to be 
identified, and a general decoupling of the end point and contact forces. Object contact 
sensing seemed more dependent on the rale at which force was detected rather than the 
absolute level of force itself. Operators used this effect to detect objects by tapping the 
edge of the object with the probe. Although the sound of tapping could not be heard 
by the operator, the mechanical coupling of the probe and control handle had sufficient 
high frequency bandwidth to pass impulsive components of force to tlie control handle. 

This has significant implications for teleoperated systems connected electrically, rather than 
mechanically, since high bandwidth transmission over substantial distances may be more 
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difficult to achieve . Experiment 1 indicates that due to the relatively high level of friction 
present in the trlemnnipiiialnr, ambiguities due to end point friction lmd a significant effect 
on pe| fomihiH e, lefliilting in only *11 1° o of the trials ending in filirreflfi In lliofie llml failed, 

o|t^tftt<nn |f|Mitl«*d llllfr Urn ol wl>nl ll»n llilylil hr mid ll»nl |'M«hlltff | • t « n I » I y 

would not have aided in recognition, 'litis situation represents standard (elemanipulal m 
tasks in a frictional environment, with an average recognition time for the tnsk of 
seconds. Two mechanisms were tested as potential aids to object recognition. In the fust of 
these, Experiment 2, object contact information was supplied to the operator, and resulted 
in a substantial improvement in performance by allowing the operator to discount apparent 
boundary information generated by end point friction. Observed probe motion was still 
somewhat clumsy but operators reported better conceptualiistion of the letter shape due to 
noise rejection. In these tests, recognition times were dramatically reduced to an average of 
81 seconds, and nn failures in recognition were reported. 

In Experiment 3, the process of re perception jfij allowed operators to visualize object 

primitives by direct observation of the control handle, as well as the determination of the 
spatial relationship between these primitives. This also reduced object recognition times to 
an average of 265 seconds and the failures to identification, although the effect was not as 
great as the reduction in signal to noise ratio. Experiment 4, in which the operator held a 
geometrically similar probe to the one attached to the telemanipulator but stood directly in 
front of the taskboard and attempted to identify the letter, produced the best performance 
of all, with no failures and an average recognition time of only 15 seconds. 

It is not dear at this stage why the human system is more than an order of mngnitude 
better than what seems to he an equivalent merhaniral system. Hefnre discussing the dif- 
ferences between the systems, it Is useful to recall their Similarities. Ilotli syslemn do not 
have any form of tactile sensing or display and both rely only on proprioceptive feedback. 
In both cases, probing took place with the hand in the same position with respect to the 
probe, which was made of the same materials. Possible mechanisms which might account 
for the disparate results of the human and machine based systems include friction, iner- 
tia, compliance and kinematic redundancy. In all cases, the manually-coupled probe was 
able to trace the object boundary at high speed, often making little or no contact with the 
taskboard. FVom previous arguments, this is indicative of a system with very little internal 
friction, which seems to be the case with any biological system. The different frequencies of 
mechanical vibration generated by taskboard and object contacts were clearly discernnble 
with the manually held probe, but were absent with the mechanical system presumably 
due to mechanical filtering by the manipulator and transmission structures. The mechan- 
ical probe acceleration was less than the manual probe due to the considerable inertia of 
the manipulator. This precludes rapid tracing, and results in loss of contact when a sharp 
corner is encountered. The slow data rate imposed by the inertial effect also degrades the 
spatial mapping of object primitives by the process outlined earlier. It was also noticed that 
the human arm could generate variable compliance in different directions relative to the 
taskboard. In the exploratory procedures observed, the probe was very compliant normal to 
the board, which also assisted in reducing the sudden build up of end point fricional forces, 
yet was stiff in any direction parallel to the board so as to generate a rapid rate of change 
in contact force if motion other than along the object boundary were to take place. This 
may have been achieved because the human arm is, by definition, anthropomorphic, while 
the manipulator operates on a terminus force control principle. Finally the manipulator 
arm kinematics are not redundant as is the human arm. This means that the human arm, 
unlike the manipulator may re-position the major limbs without changing the end effector 
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(the hand). This may provide flexibility when directional compliance is required for certain 
tasks. 

Conclusions 

Comparative tests with a telemanipulator system in common use indicate that haptic 
probing in order to identify objects belonging to a wide, though well defined set, results in 
relatively poor performance. One reason suggested for this is the effect of end-point friction 
between the probe and the taskboard on which the object is mounted. End point friction is 
in turn related to the level of internal friction present within the manipulator arm. 

Shimmy feedback Impinven pel for innuce, ami two typra of fmllint It were hi Ve*l Jgn I rd. 

The effect of re-perception assisted the conceptualization of object primitives and their 
spatial relationships to each other and resulted in a reduction in the number of rroguitiou 
failures and in the reduction of object recognition times. The availability of unambiguous 
object contact information reduced the haptic signed to noise ratio and resulted in all objects 
being recognized, again with a further reductin in recognition times. 

Results obtained using direct probing by a human operator indicated substantial improve- 
ments in performance, and it in suggested that this is the result of differences in friction, 
inertia, compliance and kinematics between the human and mechanical systems. 
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Introduction 



In this paper we are interested not so much in direct manipulation of objects, but 
in remote rnn.nipnln.lion using n. mechanical teleope.ration device. The placement of a 
telemanipulator between the operator and the task to be performed acts as a kind of 
filter to many kinds of sensory feedback, including touch, and in most cases degrades 
the operator’s ability to accomplish tasks that would be quite simple if performed 



Abstract 

In this work, we are concerned with the role that the haptic system plays in 
teleoperation i.e. the exploration and manipulation of objects by a human operator 
using a remote robot arm. The approach is to incrementaly enhance the remote 
touch-sensing capability beyond kinesthetic, force feedback to include contact data, 
and local re-perception and compare the time to identify quasi-two dimensional 
objects with that of a directly held probe. Results obtained indicated that friction 
between the remote probe and the environment made the feedback signal “noisy” 
leading to conflicting and inaccurate hypotheses by the operators. Sensory feedback 
improved the signal to noise ratio giving performance levels approaching those of 
direct, as opposed to remote, probing. 

Relavence of Work 

Experimental work is described which investigates the way humans identify ob- 
jects using the sense of touch (haptics) only, in conjunction with a remote teleoper- 
ator mechanism. The work impacts what may be achieved through telepresence in 
the control and use of remotely operated vehicles (ROV’s) particularly for 6ub-sea 
operations where visual data may not be available. 
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6. Motor control mechanisms to re-distribute the primary sensor systems 

Although the above does not provide a formal definition of haptics, it allows 
the use of terms such as haptic recognition to mean that part of the haptic system 
dealing with the cognitive or perceptive phase of touch. Similar definitions of haptic 
sensors or haptic probing follow from the above list. In our work, wo me primarily 
interested in recognizing objects using essentially proprioceptive information, with 
little emphasis on tactile data about smaller, local features. The object set to be 
recognized has to be selected carefully, since we assume that the operator has some 
previous experience with and knowledge of the objects, yet there is sufficient vari- 
ation in the set to provide a reasonably challenging task given the capabilities of 
the telemanipulator. Previous work in this area has concluded that although the 
haptic system may be U6ed to recognize three dimensional objects both accurately 
and quickly, two dimensional object recognition is accomplished less successfully. In 
the tests reported by Lederman, Klatzky and Barber (1987), raised two dimensional 
profiles were traced and recognition was attempted, resulting in poor performance. 
In other w T ork on three dimen-sional object recognition reported by Klatzky, Led- 
erman and Metzger (1985), blindfolded subjects were allowed to pick up one of a 
set of 100 common objects and attempt recognition. In these tests, good results 
were obtained with only 4% of the teste resulting in mie-classification. Lederman, 
| Klatzky and Bajcsy (1987) report other published work which supports their con- 
clusions that the haptic system is poorer at recognizing two dimensional shapes 
compared to its abilities regarding three dimensional objects. 

In the work reported here, we have investigated further the abilities of the haptic 
system to explore and recognize essentialy two dimensional shapes using a telema- 
nipulator. While acknowledging the previous work referenced above, it was felt 
that in some respects the object sets selected for the tw T o and three dimensional ex- 
ploratory testing were disparate in their familiarity to the subjects, and other factors 
may be equally significant. For example, compare the task of identifying a contour 
profile of South America with that of recognizing a toothbrush, two examples given 
in Lederman (1987). One might argue that subjects would be more familiar with a 
toothbrush than with the map which is, after all a representation of another actual 
object. Further, in handling a real object, other sensory mechanisms are at work, 
such as explorations of weight, compliance, temperature, surface texture and so on. 
In the first case of the map, an exploratory procedure would probably be employed 
which builds primitives into features, and relates features to objects as indicated 
by Stansficld (198G) except that proprioceptive data rather than tactile data would 
be spatially integrated. The acquisition of data for this task would be much slower 
than closing one’s hand around a toothbrush, and since the haptic recognition sys- 
tem works essentially with dynamically refreshed data it might be expected that 
the toothbrush would be identified first. Some significance may also be attributed 
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l,o lli c relative role of propreocepti ve and Inc.tile sensing in the two inolc n. In the 
map tracing task, tactile sensing serves only to guide the finger along the contour 
and provides no information about the object to be recognized. Tactile sensors may 
indicate that the contour is plastic or wood, smooth or rough however none of these 
are attributes of South America. For this task, proprioceptive (kinesthetic) probing 
is the dominant mode. In the second task using three dimensional objects, their size 
was selected so they could be held in the hand which suggests that tactile sensing 
dominated kinesthetic sensing. While it is not clear what the quantitative effect of 
the proproocepti ve/tactile data ratio is, the above tests seem to have substantially 
different values. An interesting experiment, perhaps offering a more appropriate 
comparison with the two dimensional profile experiments, would be to have sub- 
jects explore larger three dimensional objects such as a kitchen range, or a personal 
computer using one finger only. 

In addressing the problem of emulating the human haptic system in machines, 
much attention has been focussed on the area of tactile sensing. Machine hap- 
tics obviously has contributions from a much larger collection of components than 
just sensing. By considering the current status of teleoperation, and the efforts to 
improve this status so that the operator actually feels as though he or she is in 
the remote workplace (telepresence), some of the other mechanisms which impact 
teleoperator performance can be identified. Our research has identified four such 
mechanisms, which we propose to define as haptic variables. They directly influence 
the ability of teleoperators to perform complex tasks. These haptic variables axe (1) 
tactile sensing (2) tactile display (3) force reflectance and (4) end effector dexterity. 
Although these mechanisms are also present in the human haptic system, they are 
less variable for the purposes of experimentation. These independent variables are 
represented in Figure 1 which also indicates examples of discrete componenents of 
each technology beginning with the most simple in the periphery of the diagram and 
increasing in complexity towards the center, which is the goal of full telepresence. 
A system comprising components located near the center of the diagram would be 
expected to have capabilities approaching those of the human hand. Note that 
for mechanical teleoperator systems with man in the loop control, the definition of 
haptic variables is consistent with the definition of the components of the haptic sys- 
tem given earlier, since the human operator provides the information transmission, 
cognitive and motor control functions directly through the manipulator. 

Method 

The objective behind the experimental work reported in this section is to inves- 
tigate object recognition through remote haptic probing alone, arid to determine 
which haptic variables will produce the most significant improvement in perfor- 
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mance. In selecting the object set that operators would be asked to identify, it 
was decided to use wooden letters of the alphabet about six inches tall and one 
inch thick. For each trial, a letter was selected nt random from the set of 20 and 
attached in a random orientation to a metal taskboard. The manipulator used was 
a CRL force reflecting unit of the terminus type, which means that only forces at 
the controller handle are sensed. A schematic diagram of this manipulator is shown 
in figure 2. Force reflection is achieved mechanically through antagonistic cables 
operating each degree of freedom of the manipulator. The remote probe consisted 
of a 1/4 inch diameter steel tube about 6 inches long. The operator handle is 
constructed in the form of a pistol-type grip, A detail of the operator handle is 
Bhowil in figure 3, while the probe is shown in figure 4. D>r all tests operators wore 
earplugs and headphones connected to a pink noise source so that all audible cues 
were masked. The operator was prevented from directly viewing the taskboard and 
remote probe by a curtain. 

A total of three operators were used in each of four experiments. For each 
experiment, some twenty trials were performed. Although each operator had prior 
experience using the teleoperator system, the trials for each of the experiments were 
performed in a random sequence to equalize the rfTects of increasing familiarity with 
the equipment over the course of the experiment. The four experiments all involved 
the recognition of a randomly selected, randomly oriented letter of the alphabet, 
but differed in the sensory feedback provided to the operator during his probing. 
The specific conditions relating to each experiment were as follows. 



Experiment 1: 
Experiment 2: 
Experiment 3: 

Experiment 4: 



Force feedback only, audio and visual masking. 

As above but with edge contact feedback. 

Same as Experiment 1, but with observation of hand 
controller movement permitted. 

No telemanipulator, tracing using hand held probe 



Experiment 2 allowed the operator to observe when the remote probe was in con- 
tact with the edge of the letter. To achieve this, each wooden letter had aluminum 
foil tape wrapped around the edge, and a battery connected to it. The other battery 
terminal was connected to the remote probe so that when the probe was in contact 
with the edge of the letter a circuit was completed and a small light emitting diode 
(LED) was illuminated. The LED was placed into a small hole drilled into the mask 
worn by the operator so that edge contac confirmation was made available to him 
directly in front of his right eye. Prior to each test session, the operator was allowed 
to view a copy of the font used for the characters so that subtle differences could 
be observed, such as the difference between an M and inverted W, or an N and a 
Z. Operators were asked to provide a running commentary of what they thought 
was happening during the test. Operator comments and remote probe activity were 
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recorded onto the flame video tape. The operator wan instructed to emphasize nc- 
curacy of identification rather than speed, and to make a clear identification at the 
conclusion of the probing. Operators were told if they made an incorrect identi- 
fication, and the test continued. A time limit of ten minutes for recognition of a 
single letter was enforced, and the number of trials resulting in non-identification 
after this period was recorded. A second performance measure was the elapsed time 
required for correct identification, which was then averaged over all trials for the 
experiment. 



Results 

The results are shown in figure 5 which indicates both the average time for 
character identification and the number of times identification was not possible 
within the permitted time period. 



Discussion 

The results of Experiment 1, together with the commentary provided by oper- 
ators for the cases where no successful identification was achieved, provide insight 
into the process of teleoperation, and how performance may be enhanced. The main 
ambiguity observed by operators was the inability to distinguish contact forces be- 
tween the probe and the letter from frictional forces between the probe and the 
taskboard. This was observed during the tests when the probe appeared to be fol- 
lowing an edge quite accurately, but when a corner occurred, the probe continued to 
move in a straight line along the taskboard, constrained only by the end-point fric- 
tional forces. This is shown in figure 6 where the probe moves from A to the corner 
B, but continues in the direction C. The operator was unaware of the situation until 
the length of the perceived edge became larger than his a priori expectations about 
the longest real edge, and a re-exploration of the taskboard occurred. This leads to 
the concept of a signal-to-noise ratio for haptic data where the signal in this case is 
the probe-letter contact force and the noise is the probe-taskboard frictional force. 
In situations such as Experiment 1 where the signal is relatively small compared 
to the noise, many ambiguities arise making the identification of simple primitives 
such as edges or corners difficult. Frequently, even contact itself was incorrectly 
assumed to have occurred and haptic probing took place totally in the presence of 
noise only. 

The generation of end-point frictional forces at the probe is closely linked to the 
levels of internal frictional forces within the telemanipulator itself, as indicated by 
the following example. Suppose the control handle can be moved in any direction 
in space, but to cause such movement a static frictional force of say 5N has to be 
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overcome. No obstacle in the path of the remote probe can be detected until a 
force greater than 5N is exerted. If one considers the general strategy of probing 
a letter, the operator attempts to conceptualize the location and orientation of the 
taskboard and then tries to move the probe lightly over the board until contact with 
the letter is made. Then, still minimizing the contact force between the probe and 
the taskboard, the probe is traced around the letter building up individual primitives 
into features and spatially mapping them into a recognizable object. Light contact 
with the taskboard is required to minimize the generation of end point frictional 
forces thereby increasing the signal to noise ratio during the probing phase. The 
Internal friction of the manipulator however determines the magnitude of the friction 
force at the probe tip through the equation of static friction: 

F = fiR 

where F is the frictional force along the plane of the taskboard, /x is the coefficient 
of static friction for the pair of materials comprising the task board and probe, and 
R is the force applied normal to the task board by the operator. If P is the internal 
static frictional force of the manipulator in a direction normal to the task board, it 
can be seen that 



F > fiP 

Higher internal frictional forces therefore require larger contact forces in order 
to distinguish end point friction from object contact, and the search is characterised 
by clumsy, sometimes sudden movements when the probe leaves the object. Large 
excursions of the probe inhibit accurate relocation on the object and generally 
degrade the spatial mapping of primitives already identified. 

Lower frictional forces allow delicate probing to take place, small, subtle fea- 
tures to be identified, and a general decoupling of the end point and contact forces. 
A somewhat different effect is due to internal compliance of the telemanipulator. 
Operators felt that if the remote probe came into contact with a rigid object and 
then deformed under the action of a form of torsional stiffness, the control handle 
did not reflect the existence of the object very precisely. Object contact sensing 
seemed more dependent on the rate at which force was detected rather than the 
absolute level of force itself. Operators used this effect to detect objects by tapping 
the edge of the object with the probe. Although the sound of tapping could not 
be heard by the operator, the mechanical coupling of the probe and control han- 
dle had sufficient high frequency bandwidth to pass impulsive component; of force 
to the control handle. This has significant implications for teleoperated systems 
connected electrically, rather than mechanically, since high bandwidth transmission 
over substantial distances may be more difficult to achieve. Experiment 1 indicates 
that due lo the relatively liigli level of friction present in the leleirianipulator, am- 
biguities due to end point friction had a significant effect on performance, resulting 
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in only 40% of the trials ending in success. In those that failed, operators reported 
little idea of what the letter might be and that further probing probably would 
not lmve aided In recognition. Tliifl situation represents standard lelemanipnlator 
tasks in a frictional environment, with an average recognition time for the task of 
390 seconds. Two mechanisms were tested as potential aids to object recognition. 
In the first of these, Experiment 2, object contact information was supplied to the 
operator, and resulted in a substantial improvement in performance by allowing 
the operator to discount apparent boundary information generated by end point 
friction. Observed probe motion was still somewhat clumsy but operators reported 
better conceptualization of the letter shape due to noise rejection. In these tests, 
recognition times were dramatically reduced to an average of 84 seconds, and rio 
failures in recognition were reported. 

In Experiment 3, the process of re-perception (Kearst 1978) allowed operators 
to visualize object primitives by direct observation of the control handle, as well 
as the determination of the spatial relationship between these primitives. This also 
reduced object recognition times to an average of 265 seconds and the failures to 
identification, although the effect was not as great as the reduction in signal to noise 
ratio. Experiment 4, in which the operator held a geometrically similar probe to 
the one attached to the telemanipulator but stood directly in front of the taskboard 
and attempted to identify the letter, produced the best performance of all, with no 
failures and an average recognition time of only 15 seconds. 

In interpreting the results of these experiments, it is worthwhile remembering 
that one of the dominant effects in haptic recognition is the speed at which infor- 
mation is gathered and retained by the sensory and cognitive systems. The ability 
therefore to distinguish signal from noise is useful by itself, but since it will allow 
exploratory procedures to be performed much more rapidly, it will increase the 
likelihood of recognition. A nonlinear effect may be expected in the reduction of 
recognition times as the haptic process is aided by sensory feedback due to this rate 
of signal acquisition. 

It is not clear at this stage why the human system is more than an order of 
magnitude better than what seems to be an equivalent mechanical system. Before 
discussing the differences between the systems, it is useful to recall their similarities. 
Both systems do not have any form of tactile sensing or display and both rely only 
on proprioceptive feedback. In both cases, probing took place with the hand in the 
same position with respect to the probe, which was made of the same materials. 
Possible mechanisms which might account for the disparate results of the human 
and machine based systems include friction, inertia, compliance and kinematic re- 
dundancy. In all cases, the manually-coupled probe was able to trace the object 
boundary at high speed, often making little or no contact with the taskboard. FVom 
previous arguments, this is indicative of a system with very little internal friction, 
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which seems to be the case with any biological system. The different frequencies 
of mechanical vibration generated by taskboard and object contacts were clearly 
discernable with the manually held probe, but were absent with the mechanical 
system presumably due to mechanical filtering by the manipulator and transmis- 
sion structures. The mechanical probe acceleration was less than the manual probe 
due to tbe considerable inertia of the manipulator. This precludes rapid tracing, 
and results in loss of contact when a sharp corner is encountered. The slow data 
rate imposed by the inertial effect also degrades the spatial mapping of object prim- 
itives by the process outlined earlier. It was also noticed that the human arm could 
generate variable compliance in different directions relative to the taskboard. In 
the exploratory procedures observed, the probe was very compliant normal to the 
board, which also assisted in reducing the sudden build up of end point fricional 
forces, yet was stiff in any direction parallel to the board so ns to generate a rapid 
rate of change in contact force if motion other than along the object boundary were 
to take place. This may have been achieved because the human arm is, by defini- 
tion, anthropomorphic, while the manipulator operates on a terminus force control 
principle. Finally the manipulator arm kinematics are not redundant as is the hu- 
man arm. This means that the human arm, unlike the manipulator may re-position 
the major limbs without changing the end effector (the hand). This may provide 
flexibility when directional compliance is required for certain tasks. 

Conclusions 

Comparative tests with a telemanipulator system in common use indicate that 
haptic probing in order to identify objects belonging to a wide, though well defined 
set, results in relatively poor performance. One reason suggested for this is the effect 
of end-point friction between the probe and the taskboard on which the object is 
mounted. End point friction is in turn related to the level of internal friction present 
within the manipulator arm. 

Sensory feedback improves performance, and two types of feedback were investi- 
gated. The effect of re-perception assisted the conceptualization of object primitives 
and their spatial relationships to each other and resulted in a reduction in the num- 
ber of rcognition failures and in the reduction of object recognition times. The 
availability of unambiguous object contact information reduced the haptic signal 
to noise ratio and resulted in all objects being recognized, again with a further 
reductin in recognition times. 

Results obtained using direct probing by a human operator indicated substantial 
improvements in performance, and it is suggested that this is the result of differences 
in friction, inertia, compliance and kinematics between the human and mechanical 
systems. 
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FIGURE 1 VARIABLES OF HAPTIC PERCEPTION 



1 



MASTER 

ARM 




FIGURE 2 FORCE REFLECTING TELEMANIFULATOR 
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FIGURE 6 EFFECT OF FRICTION ON TROBE TRACKING 
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APPENDIX C 
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